Supplementary Materials: R Scripts for Shadow CAT Simulation
================

# Overview

This repository contains all necessary materials to replicate the
simulation studies reported in the manuscript:

> *Improving Item Pool Utilization for Health Professions Examinations
> Under Variable-length CAT designs: A Shadow-Test Approach*

The folder is organized as follows:

- `data/`: Contains the input files required for the simulation.
  - `Item_Bank.csv`: The simulated 360-item item pool with 3PL
    parameters.
  - `Content_Requirement.csv`: The test blueprint defining the minimum
    and maximum number of items per content category.
- `R/`: Contains all R source code functions called by the main
  simulation scripts.
  - `shadowCAT.R`: Core functions for running the CAT simulations for a
    group of examinees.
  - `shadow_sim.R`: Functions that simulate a single CAT administration
    for one examinee (Stage 1 and Stage 2).
  - `evaluation.R`: Functions to calculate the evaluation metrics (e.g.,
    accuracy, pool utilization, efficiency).
- `Rscript/`: Contains the main R scripts to execute the simulations.
- `ShadowCAT.Rproj`: An RStudio project file to easily set up the
  working environment.
- `README.md`: This file, providing guidance on how to use the
  supplementary materials.
- `output/`: A directory that will be automatically created to store
  simulation results.

# Simulation Scripts

Two main scripts are provided in the `Rscript/` folder to reproduce the
results for the two simulation conditions reported in the manuscript.

### 1. Overall Sample Simulation

**File**: `1.Simulation_Overall-Sample-Condition.R` - **Purpose**: Runs
the simulation for the “Overall Sample” condition. - **Examinees**:
50,000 simulees with true abilities sampled from a standard normal
distribution, N(0, 1). - **Output**: Generates `.rds` files for the raw
simulation object and a final summary table (`.csv`) for each of the
five item selection algorithms.

### 2. Conditional Sample Simulation

**File**: `2.Simulation_Conditional-Sample-Condition.R` - **Purpose**:
Runs the simulation for the “Conditional Samples” condition. -
**Examinees**: Simulates 500 examinees at each of 13 discrete ability
points from -3.0 to 3.0. - **Output**: Generates `.rds` files containing
the processed evaluation metrics (accuracy and efficiency) for each
algorithm, which are used to create the figures in the manuscript.

# How to Run the Simulation

To replicate the simulations, please follow these steps:

### 1. Set up the Environment

Open the `ShadowCAT.Rproj` file in RStudio. This will automatically set
the project’s root directory, ensuring that all file paths work
correctly.

### 2. Install Required Packages

Run the following command in the R console to install all necessary
packages. The `parallel` package is part of the R base installation and
does not need to be installed separately.

``` r
install.packages(c("irtQ", "lpSolveAPI", "pbapply", "tidyverse", "here", "rowr"))
```
